Main

In your final repo, there should be an R markdown file that organizes all computational steps for evaluating your proposed Facial Expression Recognition framework.

This file is currently a template for running evaluation experiments. You should update it according to your codes but following precisely the same structure.

rr if(!require()){ install.packages() BiocManager::install() } if(!require(.matlab)){ install.packages(.matlab) } if(!require()){ install.packages() } if(!require()){ install.packages() } if(!require()){ install.packages() } if(!require(2)){ install.packages(2) } if(!require()){ install.packages() } if(!require()){ install.packages() } if(!require()){ install.packages() } if(!require(1071)){ install.packages(1071) } if(!require()){ install.packages() } if(!require()){ install.packages() } library(R.matlab) library(readxl) library(dplyr) library(EBImage) library(ggplot2) library(caret) library(glmnet) library(WeightedROC) library(e1071) library(xgboost) library(randomForest)

Step 0 set work directories

set.seed(2020)
# setwd("~/Project3-FacialEmotionRecognition/doc")
# here replace it with your own path or manually set it in RStudio to where this rmd file is located. 
# use relative path for reproducibility

Provide directories for training images. Training images and Training fiducial points will be in different subfolders.

rr train_dir <- ../data/train_set/ # This will be modified for different data sets. train_image_dir <- paste(train_dir, /, sep=\) train_pt_dir <- paste(train_dir, /, sep=\) train_label_path <- paste(train_dir, .csv, sep=\)

Step 1: set up controls for evaluation experiments.

In this chunk, we have a set of controls for the evaluation experiments.

• (T/F) cross-validation on the training set
• (T/F) reweighting the samples for training set
• (number) K, the number of CV folds
• (T/F) process features for training set
• (T/F) run evaluation on an independent test set
• (T/F) process features for test set

rr run.cv <- FALSE # run cross-validation on the training set run.cv.baseline <- FALSE # run cross-validation on the gbm baseline sample.reweight <- TRUE # run sample reweighting in model training K <- 5 # number of CV folds run.feature.train <- TRUE # process features for training set run.test <- TRUE # run evaluation on an independent test set run.feature.test <- TRUE # process features for test set run.cv.svm <- FALSE run.test.svm <- TRUE

Using cross-validation or independent test set evaluation, we compare the performance of models with different specifications. In this Starter Code, we tune parameter lambda (the amount of shrinkage) for logistic regression with LASSO penalty.

rr lmbd = c(1e-3, 5e-3, 1e-2, 5e-2, 1e-1) model_labels = paste(Penalty with lambda =, lmbd) #gbm parameters tuning: n.trees = c(10,50,100,200) shrinkage = c(0.01,0.05,0.1,0.15) #svm parameters tuning: cost = c(0.0000001,0.000001,0.00001,0.0001,0.001,0.01,1) model_labels_svm = paste(with cost =, cost) #xgboost parameters tuning params <- list(booster = , objective = :logistic, eta=0.3, gamma=0, max_depth=6, min_child_weight=1, subsample=1, colsample_bytree=1) #random forest parameters tuning ntrees <- 128 #according to a paper by Thais Mayumi Oshiro, Pedro Santoro Perez, and Jos ́e Augusto Baranauska, # the AUC gain for increasing number of trees is minimal after 128, # after observing a multitude of their datasets

Step 2: import data and train-test split

rr #train-test split info <- read.csv(train_label_path) n <- nrow(info) n_train <- round(n*(4/5), 0) train_idx <- sample(info\(Index, n_train, replace = F) test_idx <- setdiff(info\)Index, train_idx)

If you choose to extract features from images, such as using Gabor filter, R memory will exhaust all images are read together. The solution is to repeat reading a smaller batch(e.g 100) and process them.

rr n_files <- length(list.files(train_image_dir)) image_list <- list() for(i in 1:100){ image_list[[i]] <- readImage(paste0(train_image_dir, sprintf(%04d, i), .jpg)) }

Fiducial points are stored in matlab format. In this step, we read them and store them in a list.

rr #function to read fiducial points #input: index #output: matrix of fiducial points corresponding to the index readMat.matrix <- function(index){ return(round(readMat(paste0(train_pt_dir, sprintf(%04d, index), .mat))[[1]],0)) } #load fiducial points fiducial_pt_list <- lapply(1:n_files, readMat.matrix) save(fiducial_pt_list, file=../output/fiducial_pt_list.RData)

Step 3: construct features and responses

• The follow plots show how pairwise distance between fiducial points can work as feature for facial emotion recognition.

• In the first column, 78 fiducials points of each emotion are marked in order.
• In the second column distributions of vertical distance between right pupil(1) and right brow peak(21) are shown in histograms. For example, the distance of an angry face tends to be shorter than that of a surprised face.

• The third column is the distributions of vertical distances between right mouth corner(50) and the midpoint of the upper lip(52). For example, the distance of an happy face tends to be shorter than that of a sad face.

Figure1

feature.R should be the wrapper for all your feature engineering functions and options. The function feature( ) should have options that correspond to different scenarios for your project and produces an R object that contains features and responses that are required by all the models you are going to evaluate later.

• feature.R
• Input: list of images or fiducial point
• Output: an RData file that contains extracted features and corresponding responses

rr source(../lib/feature.R) tm_feature_train <- NA if(run.feature.train){ tm_feature_train <- system.time(dat_train <- feature(fiducial_pt_list, train_idx)) save(dat_train, file=../output/feature_train.RData) }else{ load(file=../output/feature_train.RData) } tm_feature_test <- NA if(run.feature.test){ tm_feature_test <- system.time(dat_test <- feature(fiducial_pt_list, test_idx)) save(dat_test, file=../output/feature_test.RData) }else{ load(file=../output/feature_test.RData) }

Step 4: Train a classification model with training features and responses

Call the train model and test model from library.

train.R and test.R should be wrappers for all your model training steps and your classification/prediction steps.

• train.R
• Input: a data frame containing features and labels and a parameter list.
• Output:a trained model
• test.R
• Input: the fitted classification model using training data and processed features from testing images
• Input: an R object that contains a trained classifier.

• Output: training model specification

• In this Starter Code, we use logistic regression with LASSO penalty to do classification.

rr source(../lib/train.R) source(../lib/test.R) source(../lib/train_gbm.R) source(../lib/test_gbm.R) source(../lib/train_SVM.R) source(../lib/test_SVM.R) source(../lib/fit_train_xgboost.R) source(../lib/fit_train_randomforest.R)

Model selection with cross-validation

• Do model selection by choosing among different values of training model parameters.

Baseline Model * Baseline/GBM

rr feature_train = as.matrix(dat_train[, -6007]) label_train = as.integer(dat_train$label)

source(../lib/cross_validation.R) source(../lib/cross_validation_SVM.R) source(../lib/cv_gbm.R)

if(run.cv.baseline){

mean_error_cv <- matrix(0, nrow = length(n.trees), ncol = length(shrinkage)) sd_error_cv <- matrix(0, nrow = length(n.trees), ncol = length(shrinkage)) mean_auc_cv <- matrix(0, nrow = length(n.trees), ncol = length(shrinkage)) sd_auc_cv <- matrix(0, nrow = length(n.trees), ncol = length(shrinkage))

for(i in 1:length(n.trees)){ cat(.trees =, n.trees[i],\n) for(k in 1:length(shrinkage)){ cat(=, shrinkage[k],\n)

res_cv_gbm <- cv_gbm(features = feature_train, labels = label_train, K, n.trees = n.trees[i],shrinkage = shrinkage[k],reweight = sample.reweight)

mean_error_cv[i,k]<-res_cv_gbm[1]
 sd_error_cv[i,k]<-res_cv_gbm[2]
  mean_auc_cv[i,k]<-res_cv_gbm[3]
   sd_auc_cv[i,k]<-res_cv_gbm[4]
  
save(mean_error_cv, file=\../output/mean_error_cv.RData\)
save(sd_error_cv, file=\../output/sd_error_cv.RData\)
save(mean_auc_cv, file=\../output/mean_auc_cv.RData\)
save(sd_auc_cv, file=\../output/sd_auc_cv.RData\) 
}}

} else{ load(../output/mean_error_cv.RData) load(../output/sd_error_cv.RData)
load(../output/mean_auc_cv.RData) load(../output/sd_auc_cv.RData) }

if(run.cv){ res_cv <- matrix(0, nrow = length(lmbd), ncol = 4) for(i in 1:length(lmbd)){ cat(= , lmbd[i], \n) res_cv[i,] <- cv.function(features = feature_train, labels = label_train, K, l = lmbd[i], reweight = sample.reweight) save(res_cv, file=../output/res_cv.RData) } }else{ load(../output/res_cv.RData) }

• SVM

rr if(run.cv.svm){ res_cv_svm <- matrix(0, nrow = length(cost), ncol = 4) for(i in 1:length(cost)){ cat(= , cost[i], \n) res_cv_svm[i,] <- svm_cv(features = feature_train, labels = label_train, K, cost=cost[i], reweight = sample.reweight) save(res_cv_svm, file=../output/res_cv_svm.RData) } }else{ load(../output/res_cv_svm.RData) }

Improved Model * XGBoost

rr source(../lib/xgboost_cv.R) feature_train = as.matrix(dat_train[, -6007]) label_train = as.integer(dat_train$label) label_train_xgb <- label_train label_train_xgb[label_train_xgb == 2] <- 0 set_rounds <- 50 K <- 5 new_params <- cv_xgboost(params, feature_train, label_train_xgb, set_rounds, K)

[1] train-error:0.102605+0.007682   test-error:0.221244+0.013096 
[11]    train-error:0.005313+0.001335   test-error:0.178322+0.017084 
[21]    train-error:0.000000+0.000000   test-error:0.173332+0.013634 
[31]    train-error:0.000000+0.000000   test-error:0.169574+0.016910 
[41]    train-error:0.000000+0.000000   test-error:0.171246+0.015282 
[50]    train-error:0.000000+0.000000   test-error:0.169996+0.013158 

rr library(tidyr)

df_mean_error=data.frame(mean_error_cv)%>% setNames(shrinkage)%>% mutate(n.trees=n.trees)%>% gather(shrinkage,mean_error,0.01:0.15)

df_sd_error=data.frame(sd_error_cv)%>% setNames(shrinkage)%>% mutate(n.trees=n.trees)%>% gather(shrinkage,sd_error,0.01:0.15)

df_mean_auc=data.frame(mean_auc_cv)%>% setNames(shrinkage)%>% mutate(n.trees=n.trees)%>% gather(shrinkage,mean_auc,0.01:0.15)

df_sd_auc=data.frame(sd_auc_cv)%>% setNames(shrinkage)%>% mutate(n.trees=n.trees)%>% gather(shrinkage,sd_auc,0.01:0.15)

res_cv_gbm <- df_mean_error%>%mutate(sd_error=df_sd_error\(sd_error, mean_auc=df_mean_auc\)mean_auc, sd_auc=df_sd_auc$sd_auc) save(res_cv_gbm,file = ../output/res_cv_gbm.RData)

Visualize cross-validation results.

rr load(../output/res_cv_gbm.RData)

if(run.cv.baseline){ p1 <- res_cv_gbm %>% ggplot(aes(x = n.trees, y = mean_error, ymin = mean_error - sd_error, ymax = mean_error +sd_error)) + geom_crossbar() + facet_wrap(~shrinkage) + theme(axis.text.x = element_text(angle = 90, hjust = 1))

p2 <- res_cv_gbm %>% ggplot(aes(x = n.trees, y = mean_auc, ymin = mean_auc - sd_auc, ymax = mean_auc + sd_auc)) + facet_wrap(~shrinkage) + geom_crossbar() + theme(axis.text.x = element_text(angle = 90, hjust = 1))

print(p1) print(p2) }

res_cv_svm <- as.data.frame(res_cv_svm) colnames(res_cv_svm) <- c(_error, _error, _AUC, _AUC) res_cv_svm$k = as.factor(cost) run.cv.svm <- TRUE if(run.cv.svm){ p1 <- res_cv_svm %>% ggplot(aes(x = as.factor(cost), y = mean_error, ymin = mean_error - sd_error, ymax = mean_error + sd_error)) + geom_crossbar() + theme(axis.text.x = element_text(angle = 90, hjust = 1))

p2 <- res_cv_svm %>% ggplot(aes(x = as.factor(cost), y = mean_AUC, ymin = mean_AUC - sd_AUC, ymax = mean_AUC + sd_AUC)) + geom_crossbar() + theme(axis.text.x = element_text(angle = 90, hjust = 1))

print(p1) print(p2) }

• Choose the "best" parameter value for baseline model

rr best_n.trees = as.numeric(res_cv_gbm[which.min(res_cv_gbm$mean_error),1]) best_shrinkage = as.numeric(res_cv_gbm[which.min(res_cv_gbm$mean_error),2])

best_cost <- cost[which.min(res_cv_svm$mean_error)]

• Choose "best" number of trees and mtry for random forest model

rr source(../lib/rf_param_choice.R) feature_train = as.matrix(dat_train[, -6007]) label_train = as.integer(dat_train$label) init_mtry <- sqrt(ncol(dat_train)) rf_opt_tree <- param_choice_rf(feature_train = feature_train, label_train = label_train, mtry = init_mtry, ntree = ntrees)

Train different models

• Train the baseline model with the entire training set using the selected model (model parameter) via cross-validation.

rr # training weights weight_train <- rep(NA, length(label_train)) for (v in unique(label_train)){ weight_train[label_train == v] = 0.5 * length(label_train) / length(label_train[label_train == v]) } if (sample.reweight){ tm_train_baseline <- system.time(fit_train_baseline <- train_gbm(feature_train, label_train, w = weight_train,best_n.trees, best_shrinkage)) } else { tm_train_baseline <- system.time(fit_train_baseline <- train_gbm(feature_train, label_train, w = NULL, best_n.trees, best_shrinkage)) } save(fit_train_baseline, file=../output/fit_train_baseline.RData)

• Train the SVM model with the entire training set using the selected model (model parameter) via cross-validation.

rr weight_train <- rep(NA, length(label_train)) for (v in unique(label_train)){ weight_train[label_train == v] = 0.5 * length(label_train) / length(label_train[label_train == v]) } if (sample.reweight){ tm_train_svm <- system.time(fit_train_svm <-svm_train(feature_train, label_train, w = weight_train, best_cost)) } else { tm_train_svm <- system.time(fit_train_svm <-svm_train(feature_train, label_train, w = NULL, best_cost)) } save(fit_train_svm, file=../output/fit_train_svm.RData)

• Train the XGBoost model with optimal parameters

rr xgb_train_time <- system.time(fit_train_xgb <- xgboost_train(features = feature_train, labels = label_train_xgb, params = new_params, rounds = set_rounds))

[22:46:58] WARNING: amalgamation/../src/learner.cc:541: 
Parameters: { early_stop_round, silent } might not be used.

  This may not be accurate due to some parameters are only used in language bindings but
  passed down to XGBoost core.  Or some parameters are not used but slip through this
  verification. Please open an issue if you find above cases.


[1] train-error:0.116250 
[2] train-error:0.091250 
[3] train-error:0.080833 
[4] train-error:0.070417 
[5] train-error:0.059167 
[6] train-error:0.050000 
[7] train-error:0.040000 
[8] train-error:0.031667 
[9] train-error:0.025833 
[10]    train-error:0.020417 
[11]    train-error:0.015000 
[12]    train-error:0.011667 
[13]    train-error:0.007500 
[14]    train-error:0.003750 
[15]    train-error:0.002500 
[16]    train-error:0.001667 
[17]    train-error:0.001667 
[18]    train-error:0.000417 
[19]    train-error:0.000000 
[20]    train-error:0.000000 
[21]    train-error:0.000000 
[22]    train-error:0.000000 
[23]    train-error:0.000000 
[24]    train-error:0.000000 
[25]    train-error:0.000000 
[26]    train-error:0.000000 
[27]    train-error:0.000000 
[28]    train-error:0.000000 
[29]    train-error:0.000000 
[30]    train-error:0.000000 
[31]    train-error:0.000000 
[32]    train-error:0.000000 
[33]    train-error:0.000000 
[34]    train-error:0.000000 
[35]    train-error:0.000000 
[36]    train-error:0.000000 
[37]    train-error:0.000000 
[38]    train-error:0.000000 
[39]    train-error:0.000000 
[40]    train-error:0.000000 
[41]    train-error:0.000000 
[42]    train-error:0.000000 
[43]    train-error:0.000000 
[44]    train-error:0.000000 
[45]    train-error:0.000000 
[46]    train-error:0.000000 
[47]    train-error:0.000000 
[48]    train-error:0.000000 
[49]    train-error:0.000000 
[50]    train-error:0.000000 

rr save(fit_train_xgb, file=../output/fit_train_xgb.RData)

• Train the Random Forest model with optimal parameters

rr load(file = ../output/res_oob_rf.RData) rf_train_time <- system.time(fit_train_randomforest <- rf_train(feature_train = feature_train, label_train = label_train, mtry = rf_opt_tree\(mtry, ntree = rf_opt_tree\)ntree)) save(fit_train_rf, file=../output/fit_train_rf.RData)

Error in save(fit_train_rf, file = \../output/fit_train_rf.RData\) : 
  目标对象‘fit_train_rf’不存在

• train PCA + LDA

rr #PCA for training features source(../lib/train_PCA.R) # make dat_train a numeric data frame dat_train.new <- matrix(0, ncol = ncol(dat_train) - 1, nrow = nrow(dat_train)) for (i in 1:(ncol(dat_train) - 1)) { dat_train.new[,i] <- as.numeric(dat_train[[i]]) } dat_train.new <- as.data.frame(dat_train.new) #PCA for training features tm_train_pca <- system.time(fit_train_pca <- train_pca(dat_train.new)) save(fit_train_pca, file=../output/pca_train.RData) # determine the important principle components screeplot(fit_train_pca, type = )

rr # The proportion of variance for first 300 PCs sum((fit_train_pca\(sdev[1:300])^2) / sum((fit_train_pca\)sdev)^2)

[1] 0.998993

Therefore, we can choose the first 300 principle components because they explain most of the total variance, which is around 99.9%.

rr # get the features of principle components with emotion index dat_train_pca <- data.frame(fit_train_pca$x[,1:300], emotion_idx = dat_train[,6007])

Use trained PCA model to test data

rr source(../lib/test_PCA.R) dat_test.new <- dat_test colnames(dat_test.new) <- c(colnames(dat_train.new), _idx) tm_test_pca <- system.time(dat_test.new <- test_pca(fit_train_pca, dat_test.new)) #features of testing principle components with the emotion index dat_test_pca <- data.frame(dat_test.new[,1:300], emotion_idx = dat_test[,6007]) save(dat_train_pca, file=../output/feature_train_pca.RData) save(dat_test_pca, file=../output/feature_test_pca.RData)

Apply LDA model

rr #load(../output/feature_train_pca.RData) #train LDA model source(../lib/train_LDA.R) tm_train_lda <- system.time(fit_train_lda <- train_lda(dat_train_pca))

载入程辑包：‘MASS’

The following object is masked from ‘package:dplyr’:

    select

rr save(fit_train_lda, file=../output/LDA_train.RData) #training accuracy in LDA model source(../lib/test_LDA.R) pred_train_lda <- test_lda(fit_train_lda, dat_train_pca) accu_train_lda <- mean(pred_train_lda == dat_train_pca$emotion_idx) #cat(trainig accuracy of model LDA, , accu_train_lda*100, %.) #cat(for training model LDA = , tm_train_lda[1], )

Step 5: Run test on test images

*Baseline model

rr tm_test_baseline = NA feature_test <- as.matrix(dat_test[, -6007]) if(run.test){ load(file=../output/fit_train_baseline.RData) tm_test_baseline <- system.time({label_pred_baseline <- as.integer(test_gbm(fit_train_baseline,feature_test,best_n.trees, best_shrinkage, pred.type = 'link')); prob_pred_baseline <- test_gbm(fit_train_baseline, feature_test,best_n.trees, best_shrinkage, pred.type = 'response')}) }

*SVM model

rr tm_test = NA feature_test <- as.matrix(dat_test[, -6007]) if(run.test.svm){ load(file=../output/fit_train_svm.RData) tm_test_svm <- system.time({ label_pred_svm <- as.integer(svm_test(fit_train_svm, feature_test, pred.type = 'class')); prob_pred_svm <- svm_test(fit_train_svm, feature_test, pred.type = 'response')}) }

*XGBoost

rr tm_test_xgb = NA feature_test <- as.matrix(dat_test[, -6007]) if(run.test){ load(file=../output/fit_train_xgb.RData) tm_test_xgb <- system.time({label_pred_xgb <- predict(fit_train_xgb, feature_test, pred.type = 'class'); label_pred_xgb[label_pred_xgb >= 0.5] <- 1; label_pred_xgb[label_pred_xgb < 0.5] <- 0; prob_pred_xgb <- predict(fit_train_xgb, feature_test, pred.type = 'response')}) }

*Random Forest

rr tm_test_rf = NA feature_test <- as.matrix(dat_test[, -6007]) if(run.test){ load(file=../output/fit_train_rf.RData) tm_test_rf <- system.time({label_pred_rf <- predict(fit_train_randomforest, feature_test, pred.type = 'class'); prob_pred_rf <- predict(fit_train_randomforest, feature_test, type = 'prob')}) }

*LDA

rr source(../lib/test_LDA.R) #load(../output/feature_test_pca.RData) load(file=../output/LDA_train.RData) tm_test_lda <- system.time(pred_lda <- test_lda(fit_train_lda, dat_test_pca))

Evaluation

rr ## reweight the test data to represent a balanced label distribution label_test <- as.integer(dat_test$label) weight_test <- rep(NA, length(label_test)) for (v in unique(label_test)){ weight_test[label_test == v] = 0.5 * length(label_test) / length(label_test[label_test == v]) }

accu_baseline <- mean(label_pred_baseline == label_test) tpr.fpr.baseline <- WeightedROC(prob_pred_baseline, label_test, weight_test) auc_baseline <- WeightedAUC(tpr.fpr.baseline)

cat(accuracy of model GBM: with n.trees=,best_n.trees,shrinkage =, best_shrinkage, , accu_baseline*100, %.) cat(AUC of model GBM: with n.trees=, best_n.trees,shrinkage =, best_shrinkage, , auc_baseline, .)

rr label_test <- as.integer(dat_test$label) weight_test <- rep(NA, length(label_test)) for (v in unique(label_test)){ weight_test[label_test == v] = 0.5 * length(label_test) / length(label_test[label_test == v]) }

accu_svm <- mean(label_pred_svm == label_test) tpr.fpr.svm <- WeightedROC(prob_pred_svm, label_test, weight_test) auc_svm <- WeightedAUC(tpr.fpr.svm) cat(accuracy of model:, model_labels_svm[which.min(res_cv_svm$mean_error)], , accu_svm*100, %.) cat(AUC of model:, model_labels_svm[which.min(res_cv_svm$mean_error)], , auc_svm, .)

rr label_test <- as.integer(dat_test$label) label_test_xgb <- label_test label_test_xgb[label_test_xgb==2] = 0 accu_xgb <- mean((label_pred_xgb == label_test_xgb)) tpr.fpr_xgb <- WeightedROC(prob_pred_xgb, label_test_xgb) auc_xgb <- WeightedAUC(tpr.fpr_xgb)

cat(accuracy of the XGBoost model:, , accu_xgb*100, %.) cat(AUC of the XGBoost model:, , auc_xgb, .)

rr label_test <- as.integer(dat_test$label) accu_rf <- mean(label_pred_rf == label_test) tpr.fpr.rf <- WeightedROC(prob_pred_rf[,2], label_test) auc_rf <- WeightedAUC(tpr.fpr.rf)

cat(accuracy of the Random Forest model:, , accu_rf*100, %.) cat(AUC of the Random Forest model:, , auc_rf, .)

rr accu_lda <- mean(dat_test_pca\(emotion_idx == pred_lda) label_lda <- as.numeric(dat_test_pca\)emotion_idx) tpr.fpr.lda <- WeightedROC(as.numeric(pred_lda), label_lda) auc_lda <- WeightedAUC(tpr.fpr.lda) cat(accuracy of the PCA+LDA model:, , accu_lda*100, %.)

The accuracy of the PCA+LDA model: is 82.33333 %.

rr cat(AUC of the PCA+LDA model:, , auc_lda, .)

The AUC of the PCA+LDA model: is 0.6773504 .

Summarize Running Time

Prediction performance matters, so does the running times for constructing features and for training the model, especially when the computation resource is limited.

rr cat(for constructing training features=, tm_feature_train[1], ) cat(for constructing testing features=, tm_feature_test[1], ) cat(for training model=, tm_train_baseline[1], ) cat(for testing model=, tm_test_baseline[1], )

rr cat(for constructing training features=, tm_feature_train[1], ) cat(for constructing testing features=, tm_feature_test[1], ) cat(for training model=, tm_train_svm[1], ) cat(for testing model=, tm_test_svm[1], )

rr cat(for constructing training features=, tm_feature_train[1], ) cat(for constructing testing features=, tm_feature_test[1], ) cat(for training model=, xgb_train_time[1], ) cat(for testing model=, tm_test_xgb[1], )

rr cat(for constructing training features=, tm_feature_train[1], ) cat(for constructing testing features=, tm_feature_test[1], ) cat(for training random forest model=, rf_train_time[1], ) cat(for testing random forest model=, tm_test_rf[1], )

rr cat(for constructing training features=, tm_feature_train[1], )

Time for constructing training features= 1.402 s 

rr cat(for constructing testing features=, tm_feature_test[1], )

Time for constructing testing features= 0.3 s 

rr cat(for training PCA =, tm_train_pca[1], )

Time for training PCA = 325.249 s 

rr cat(for testing PCA =, tm_test_pca[1], )

Time for testing PCA = 8.124 s 

rr cat(for training model LDA = , tm_train_lda[1], )

Time for training model LDA =  1.927 s 

rr cat(for testing model LDA = ,tm_test_lda[1], )

Time for testing model LDA =  0.033 s 

Reference

• Du, S., Tao, Y., & Martinez, A. M. (2014). Compound facial expressions of emotion. Proceedings of the National Academy of Sciences, 111(15), E1454-E1462.